Adaptive Agents for Artificial Life Domain

Adaptation and real time behavior becomes a necessity for agents that want to survive in Artificial Life environments. Agents are autonomous and therefore some form of unsupervised learning has to be used to improve and adapt their behavior in time. The neurodynamic reinforcement learning approach – Q-learning where Q-factors are represented using neural networks, overcomes problems of the classical unsupervised algorithms. These problems are that the reinforcement is usually obtained only at the end of sequence of steps and that the action-state space is very huge and therefore some form of generalization is necessary. This research that is mainly focused on design of agent architectures for Artificial Life introduces the concepts of parallel and dynamic Q-spaces, where the learned Q-factors are divided into separate actionstate spaces that correspond to independent behaviors. Q-factors are then combined across these spaces based on relationship among higher level behaviors. Once learned the Q-factors are assigned with priority that can change in correspondence to agent dynamics and changes in the environment. A simulated artificial life environment with Artificial Life agents has been used as a test bed. The research is being conducted in the Artificial Life domain, however we believe that the proposed techniques will be useful in different domains also. Key-Words: Adaptivity, Agents, Artificial Life, Ethology, Q-Learning, Reinforcement Learning, Robotics